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ABSTRACT 

The goal of this paper is to compare the accuracy of two approximate confidence interval 
estimators for the Bernoulli parameter p. The approximate confidence intervals are based 
on the normal and Poisson approximations to the binomial distribution. Charts are given to 
indicate which approximation is appropriate for certain sample sizes and point estimators. 
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1 Introduction 


There is conflicting advice concerning the sample size necessary to use the normal 
approximation to the binomial distribution. For example, a sampling of textbooks 
recommend that the normal distribution be used to approximate the binomial distri- 
bution when: 

• np and n(l — p) are both greater than 5 (see [1], page 211, [5], page 245, [7], 
page 304, [9], page 148, [16], page 497, [17], page 161) 

• p ± 2 lies in the interval (0, 1) (see [15], page 242, [12], page 299) 

• np(l — p) > 10 (see [13], page 171) 

• np(l — p) > 9 (see [1], page 158). 

Many other textbook authors give no specific advice concerning when the normal 
approximation should be used. To complicate matters further, most of this advice 
concerns using these approximations to compute probabilities. Whether these same 
rules of thumb apply to confidence intervals is seldom addressed. The Poisson ap- 
proximation, while less popular than the normal approximation to the binomial, is 
useful for large values of n and small values of p. The same sampling of textbooks 
recommend that the Poisson distribution be used to approximate the binomial dis- 
tribution when n > 20 and p < 0.05 or n > 100 and np < 10 (see [8], page 177, [5], 
page 204). 

Let X\,X 2 ,...,X n be iid Bernoulli random variables with unknown parameter 
p and let Y = Y^=i X* be a binomial random variable with parameters n and p. 
The maximum likelihood estimator for p is p = which is unbiased and consistent. 
The interest here is in confidence interval estimators for p. In particular, we want 
to compare the approximate confidence interval estimators based on the normal and 
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Poisson approximations to the binomial distribution. Determining a confidence in- 
terval for p when the sample size is large using approximate methods is often needed 
in simulations with a large number of replications and in polling. 

Computing probabilities using the normal and Poisson approximations is not con- 
sidered here since work has been done on this problem. Ling [11] suggests using 
a relationship between the cumulative distribution functions of the binomial and F 
distributions to compute binomial probabilities. Ghosh [6] compares two confidence 
intervals for the Bernoulli parameter based on the normal approximation to the bi- 
nomial distribution. Schader and Schmid [14] compare the maximum absolute error 
in computing the cumulative distribution function for the binomial distribution us- 
ing the normal approximation with a continuity correction. They consider the two 
rules for determining whether the approximation should be used: np and n(l — p) 
are both greater than 5, and np{ 1 — p) > 9. Their conclusion is that the relationship 
between the maximum absolute error and p is approximately linear when considering 
the smallest possible sample sizes to satisfy the rules. 

Concerning work done on confidence intervals for p, Blyth [2] has compared five 
approximate one-sided confidence intervals for p based on the normal distribution. 
In addition, he uses the F distribution to reduce the amount of time necessary to 
compute an exact confidence interval. Using an arcsin transformation to improve the 
confidence limits is considered by Chen [4]. 

2 Confidence Interval Estimators for p 

Two-sided confidence interval estimators for p can be determined with the aid of 
numerical methods. One-sided confidence interval estimators are analogous. Let 
Pl < P < Pu be an “exact” (see [2]) confidence interval for p. For y = l,2, ...,n— 1, 
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the lower limit pi satisfies 


E ( l ) pio - «.)*** = «/2 

where 3 / is the observed value of the random variable K and a is the nominal coverage 
of the confidence interval (see, for example, [10], page 279). For y = 1,2, . . . ,n — 1, 
the upper limit py satisfies 

t(l) Pu( 1 - w)-‘ = “/2- 

This confidence interval requires numerical methods to determine pi and py and 
takes longer to calculate as n increases. This interval will be used as a basis to 
check the approximate bounds reviewed later in this section. A figure showing the 
coverage probabilities for bounds of this type is shown in Blyth [2]. Following a 
derivation similar to his, a faster way to determine the lower and upper limits can 
be determined. Let W\, W 2 , . . . , W n be iid U(0, 1) random variables. Let Y be the 
number of the Wj’s that are less than p. Hence Y is binomial with parameters n and 
p. Using a result from page 233 of Casella and Berger [3], the order statistic W = VF( v ) 
has the beta distribution with parameters y and n — y + 1. Since the events Y > y 
and W < p are equivalent, P[Y > y] (which is necessary for determining pi) can be 
calculated by 


P(Y > y) 


P(W < p) 

r(n + l) 

r(y)r(n — y + l) 



- w) n ~ y dw. 


Using the substitution t = and simplifying yields 


P(Y > y) 


r(n + l) , n-y + l y+1 / t y 1 ^ 

T(y)T(n-y + \y y ’ Jo («zj±l + <)n+i 


P[p2y,2{ n-y+1) < 


(» - y + j> i 
y(i-p) 
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Since this probability is equal to or/2 for a two-sided confidence interval, 


^ _ (n-y + l)p L 

^2»,2(n-v+l),l-a/2 - 


or 


PL = 


1 

1 -| , ?-y + 1 

V^2i»,2(n-»+l),l-a/2 


In a similar fashion, 

1 

PU = Y + / "3 

( y + l )^ 2 ( v +!), 2 ( n -»),<./2 

The next paragraph discusses numerical issues associated with determining these 
bounds. 

The Mathematica (see [18]) code for solving the binomial equations numerically 


is 


pi = FindRoot[ 

Sum [Binomial [n, k] * p ~ k * (1 - p) “ (n - k) , {k, y, n}] == alpha/2, 
{p, y / n} ] 


pu = FindRoot[ 

Sum [Binomial [n, k] * p * k * (1 - p) “ (n - k) , {k, 0, y}] « alpha/2, 
{p, y / n> ] 

for a given n, y and a. This code works well for small and moderate sized values 
of n. Some numerical instability occurred for larger values of n, so the well known 
relationship (Larsen and Marx [10], page 101) between the successive values of the 
probability mass function f(x) of the binomial distribution 
, (n — x + l)p ,, 

/(*) = 7j -T- f( x ~ !) x — 1, 2, . . . , n 

x(l - p ) 

was used to calculate the binomial cumulative distribution function. The Mathemat- 
ica code for determining pi and pu using the F distribution is 
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fcrit = Quantile [FRatioDistribution[2 * y, 2 * (n - y + 1)] , alpha/2] 
pl*l/(l+(n-y+l)/ (y* fcrit) ) 

fcrit * Quantile [FRatioDistribution[2 * (y + 1) , 2 * (n - y)] , 1 - alpha/2] 
pu = 1 / ( 1 + (n - y) / ( (y + 1) * fcrit) ) 

This method is significantly faster than the approach using the binomial distribution, 
but encounters difficulty with determining the F ratio quantiles for some combinations 
of n and y. 

The first approximate confidence interval is based on the normal approximation 
to the binomial. The random variable is asymptotically standard normal. 

Y np(l— p) 

Thus an approximate confidence interval for p is 



where z a / 2 is the 1 — a/2 fractile of the standard normal distribution. This approxi- 
mation works best when p = \ (e.g., political polls). It allows confidence limits that 
fall outside of the interval [0, 1]. One should also be careful when Y = 0 or Y = n 
since the confidence interval will have a width of 0. 

The second approximate confidence interval is based on the Poisson approximation 
to the binomial (see, for example, Trivedi [16], page 498). This confidence interval 
does not appear as often in textbooks as the first approximate confidence interval. 
The random variable Y is asymptotically Poisson with parameter np. Therefore, the 
exact lower bound pi satisfying 

t( n k ) 4(i - «.r‘ = «/2 

can be approximated with a Poisson lower limit ppl which satisfies 

r — m — ' 

k=y 
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or 


y-i 


i-E 

k = 0 


( nppL ) 


* p~ n PPL 


k\ 


= a/2. 


The left-hand side of this equation is the cumulative distribution function for an 
Erlang random variable with parameters nppp and y (denoted by E nppLiy ) evaluated 
at one. Consequently, 


E[E n p PLl y < 1] — a/ 2 


Since 2nppiE nr>PLt y is equivalent to a x 2 random variable with 2 y degrees of freedom, 
this reduces to 

P[xly < 2 np PL \ = a/2 


or 


PPL = 


2n 


By a similar line of reasoning, the upper 
to the binomial distribution is 


X2y,l-a/2- 

limit based on the Poisson approximation 


_ 1 2 

PPU ~ ^ X 2 (y+lW2- 

This approximation works best when p is small (e.g., reliability applications where 
the probability of failure p is small). 


3 Comparison of the Approximate Methods 

There are a multitude of different ways to compare the approximate confidence inter- 
vals with the exact values. We have decided to compute the error of an approximate 
two-sided confidence interval as the maximum error 

maxflpL - Pl\,\pu ~ Pu |} 

where pi and f>p are the approximate lower and upper bounds, respectively. This 
error is computed for all combinations of n and p. Since the definition of “success” 
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on each Bernoulli trial is arbitrary, we only consider the range 0 < p < 5. Figures 1, 
2 and 3 have mirror images for the range | < p < 1. 

Figure 1 contains a plot of n versus p for n — 2, 4, , 100 and considers the range 
0 < p < | for a maximum error of 0.01. Thus if the actual error for a particular (rc,p) 
pair is greater that 0.01, the point lands in the “Do not approximate” region. If one 
of the two approximations yields an error of less than 0.01, then the pair belongs to 
either the “Normal approximation” or “Poisson approximation” regions, depending 
on which yields a smaller error. Not surprisingly, the normal approximation performs 
better when the point estimate is closer to | and the Poisson approximation performs 
better when the point estimate is closer to 0. Both approximations perform better 
as n increases. In order to avoid any spurious discontinuities in the regions, the 
calculations were made for even values of n. The edges of the region are not smooth 
because of the discrete natures of n and p. The boundary of the approximation 
regions are those (n,p) pairs where the error is less than 0.01. If the horizontal axis 
were extended, the normal and Poisson regions would meet at approximately n — 150. 
Mathematica [18] was used for the comparisons because of its ability to hold variables 
to arbitrary precision. 

If the maximum error is relaxed to 0.04, then there are more cases where the 
approximations perform adequately. Figure 2 is analogous to Figure 1 but considers 
an error of 0.04. This figure also contains the rules of thumb associated with the 
normal and Poisson approximations to the binomial distribution. In particular, 

• the rule labeled “Rl” is a plot of p = 5 fn on the range [10, 100] corresponding 
to the normal approximation rule np > 5 and n(l — p) > 5 

• the rule labeled “R2” is a plot of p = ^ on the range [4,100] corresponding 

to the normal approximation rule p ± 2 falling in the interval (0, 1) 

• the rule labeled “R3” is a plot of p = | ^ — " 011 the ran S e [40,100] 
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corresponding to the normal approximation rule np( 1 — p) > 10 


• the rule labeled “R4” is a plot of p — | 36 ^ on the range [36,100] 

corresponding to the normal approximation rule np( 1 — p) > 9 

• the rule labeled “R5” is a plot of n > 20 and p < 0.05 or n > 100 and np < 10 
corresponding to the guideline for using the Poisson approximation. 

The n, p combinations falling above the dotted curves for rules Rl, R2, R3, and R4 
correspond to those that- would be used if the rules of thumb were followed. Clearly, 
rules R3 and R4 are significantly more conservative than Rl and R2. 

Figure 3 is a continuation of Figure 2 for sample sizes larger than n =100. Note 
that the vertical axis has been modified and the horizontal axis is logarithmic. The 
curve in the figure represents the largest value of p where the Poisson approximation 
to the binomial is superior to the normal approximation to the binomial. Since this 
relationship is linear, a rather unwieldy rule of thumb for n between 100 and 10,000 
is: use the normal approximation over the Poisson approximation if p > - . 

4 Conclusions 

Although there are a number of different variations of the calculations that have 
been conducted here (e.g., one-sided confidence intervals, different significance levels, 
different definitions of error), there are three general conclusions: 

• The traditional advice from most textbooks of using the normal and Poisson ap- 
proximations to the binomial for the purpose of computing confidence intervals 
for p should be tempered with a statement such a s: “the Poisson approximation 
should be used when n > 20 and p < 0.05 if the analyst can tolerate an error 
that may be as large as 0.04” (see Figure 2). 
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• For sample sizes larger than 150, the absolute error of either upper and lower 
confidence limit is less than 0.01 if the appropriate approximation technique 
is used. Figure 3 should be consulted for specific guidance as to whether the 
binomial or Poisson approximation is appropriate. 

• Introductory probability and statistics textbooks targeting statistics and math- 
ematics majors would benefit from including the use of the F distribution to 
find pl and py. Also, more of these texts should include the use of the Poisson 
approximation to the binomial distribution for determining interval estimates 
for p. These confidence limits only require a table look-up associated with the 
chi-square distribution and are very accurate for large n and small p. 
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